For multi-view imaging systems, FIG. 7 is a diagram showing an example of camera arrangement having a straight-line alignment, FIG. 8 is a diagram showing another example of camera arrangement having a planar arrangement, FIG. 9 is a diagram showing another example of camera arrangement having an arc arrangement, and FIG. 10 is a diagram showing another example of camera arrangement having a spherical arrangement.
The multi-view imaging systems for imaging a scene in different directions have been developed. In the multi-view imaging systems, the camera arrangement has various forms such as a one-dimensional arrangement on a straight line as shown in FIG. 7, a two-dimensional arrangement on a plane as shown in FIG. 8, an arc arrangement as shown in FIG. 9, or a spherical arrangement as shown in FIG. 10. Using such multi-view imaging systems makes it possible to archive video scenes in many directions.
In addition, there is a technique called “image synthesis” for generating image information at a virtual camera position (at which imaging is not performed) by using camera images obtained by multi-view imaging. In the image synthesis, it is assumed that a camera parameter which indicates the spatial direction in which the original video image was obtained is known.
The image synthesis can be performed by various methods. For example, there is a method of performing synthesis by estimating depth information. First, disparity information is obtained by searching corresponding points between original video images, and depth information of the relevant scene is estimated based on the disparity information. Then, depth information at the virtual camera position is estimated, and corresponding image information is generated using image information of the original cameras (see Non-Patent Document 1).
In another example of the methods, no estimation of depth information is performed, but image information at the virtual camera position is directly generated using disparity information of original images (see Non-Patent Document 2).
In another example, a plurality of camera images are used for estimating a three-dimensional model information of an object which is present in the relevant scene, and an image of the model projected from the virtual camera position is generated (see Non-Patent Document 3).
In the above-described imaging systems, imaging is generally performed using image signals having a Bayer arrangement, and the Bayer arrangement is subjected to demosaicing to obtain RGB signals or YUV signals (i.e., luminance signal Y and chrominance signals U and V).
Demosaicing is a process of estimating the three color components (RGB) of each pixel position, based on an R, G, or B signal assigned to each pixel position obtained using the Bayer arrangement (see Non-Patent Document 4). As demosaicing corresponds to increasing of the resolution of the signal of each color component from a low value to a high value, it may be performed together with a super-resolution technique (see Non-Patent Document 5).
Additionally, in the above-described imaging systems, cameras having the same resolution are generally used, however, those having different resolutions may be used. It is possible to reduce the amount of obtained image information by using a combination of a camera having a high resolution and a camera having a low resolution. In addition, the cameras may have different focusing positions or viewing angles. In such a case, even when each image signal to be obtained has the same resolution, each area which is actually imaged has an individual resolution.
To obtain an image having a high resolution based on an image having a low resolution, an enlarging method of applying an up-sampling filter to each image signal in an image having a low resolution and a super-resolution method are known.
In the enlarging method, an image signal is obtained by applying an appropriate filter to image signals in a peripheral area. In the super-resolution method, generally, information of a plurality of images having the same resolution, which were sequentially obtained, is used (see Non-Patent Document 6). First, an image having a resolution higher than the obtained image is defined as a target image. That is, each pixel position of a target to be generated is defined in advance. Next, a corresponding relationship between the obtained images is estimated, and each image signal obtained by imaging is assigned to the corresponding target pixel position, thereby obtaining image information having a high resolution.
In order to represent color signals of an image, an RGB or YUV format is known. As a uniform color space, FIG. 11 shows a Munsell color space invented by Munsell. In the Munsell color space, colors are represented using hue, value, and chroma.
The “hue” indicates tint, and has five basic hues of red (R), yellow (Y), green (G), blue (B), and purple (P). That is, 10 hues are defined together with intermediate hues such as yellow-red (YR), yellow-green (GY), blue-green (BG), blue-purple (PB), and red-purple (RP).
The “value” indicates brightness, where 0 is assigned to ideal black for complete absorption, and 10 is assigned to ideal white for complete reflection. Between them, 10 levels are defined at sensibly equal intervals. The chroma indicates vividness.
The Munsell symbol is represented as HV/C (hue·value/chroma).
When representing the Munsell color space using a chart, the hue is regularly arranged along a circumference so as to form a hue circle. For the chroma, the farther from the center, the higher the chroma, which produces a more vivid color. As an approximate space for the Munsell color space, a CIE L*a*b* space or a CIE L*u*v* space has been proposed.    Non-Patent Document 1: Keita Takahashi and Takeshi Naemura,“Layered Light-Field Rendering with Focus Measurement”, EURASIP Signal Processing: Image Communication, vol. 21, no. 6, pp. 519-530 (2006.7).    Non-Patent Document 2: M. Droese, T. Fujii and M. Tanimoto,“Ray-Space Interpolation Constraining Smooth Disparities Based On Loopy Belief Propagation”, Proc. of IWSSIP2004, pp. 247-250, Poznan, Poland, September 2004.    Non-Patent Document 3: Takashi Matsuyama, Takeshi Takai, Xiaojunn Wu, and Shohei Nobuhara, “Generation, Editing, and Visualization of 3D Video”, Proceedings of The Virtual Reality Society of Japan, Vol. 7, No. 4, pp. 521-532, 2002.12.    Non-Patent Document 4: I. Tsubaki and K. Aizawa, “Demosaicing method from pixel mixture image”, Forum on Information Science and Technology, pp. 219-222, September 2003.    Non-Patent Document 5: Tomomasa Goto and Masatoshi Okutomi, “High Resolution Color Image Reconstruction Using Raw Data of a Single Imaging Chip”, IPSJ Transactions on Computer Vision and Image Media, Vol. 45, No. SIG 8(CVIM 9), pp. 15-25, 2004.    Non-Patent Document 6: Masayuki Tanaka and Masatoshi Okutomi, “A Fast Algorithm for Reconstruction-Based Super-Resolution and Its Accuracy Evaluation”, Proceedings of IEICE, D-II vol. J88-D-II, No. 11, pp. 2200-2209, 2005.